90 research outputs found
Red teaming ChatGPT via Jailbreaking: Bias, Robustness, Reliability and Toxicity
Recent breakthroughs in natural language processing (NLP) have permitted the
synthesis and comprehension of coherent text in an open-ended way, therefore
translating the theoretical algorithms into practical applications. The large
language models (LLMs) have significantly impacted businesses such as report
summarization software and copywriters. Observations indicate, however, that
LLMs may exhibit social prejudice and toxicity, posing ethical and societal
dangers of consequences resulting from irresponsibility. Large-scale benchmarks
for accountable LLMs should consequently be developed. Although several
empirical investigations reveal the existence of a few ethical difficulties in
advanced LLMs, there is little systematic examination and user study of the
risks and harmful behaviors of current LLM usage. To further educate future
efforts on constructing ethical LLMs responsibly, we perform a qualitative
research method called ``red teaming'' on OpenAI's ChatGPT\footnote{In this
paper, ChatGPT refers to the version released on Dec 15th.} to better
understand the practical features of ethical dangers in recent LLMs. We analyze
ChatGPT comprehensively from four perspectives: 1) \textit{Bias} 2)
\textit{Reliability} 3) \textit{Robustness} 4) \textit{Toxicity}. In accordance
with our stated viewpoints, we empirically benchmark ChatGPT on multiple sample
datasets. We find that a significant number of ethical risks cannot be
addressed by existing benchmarks, and hence illustrate them via additional case
studies. In addition, we examine the implications of our findings on AI ethics
and harmal behaviors of ChatGPT, as well as future problems and practical
design considerations for responsible LLMs. We believe that our findings may
give light on future efforts to determine and mitigate the ethical hazards
posed by machines in LLM applications.Comment: Technical Repor
On the Way to SBOMs: Investigating Design Issues and Solutions in Practice
Software Bill of Materials (SBOM), offers improved transparency and supply
chain security by providing a machine-readable inventory of software components
used. With the rise in software supply chain attacks, the SBOM has attracted
attention from both academia and industry. This paper presents a study on the
practice of SBOM, based on the analysis of 4,786 GitHub discussions from 510
SBOM-related projects. Our study identifies key topics, challenges, and
solutions associated with effective SBOM usage. We also highlight commonly used
tools and frameworks for generating SBOMs, along with their respective
strengths and limitations. Our research underscores the importance of SBOMs in
software development and the need for their widespread adoption to enhance
supply chain security. Additionally, the insights gained from our study can
inform future research and development in this field
Making Python Code Idiomatic by Automatic Refactoring Non-Idiomatic Python Code with Pythonic Idioms
Compared to other programming languages (e.g., Java), Python has more idioms
to make Python code concise and efficient. Although pythonic idioms are well
accepted in the Python community, Python programmers are often faced with many
challenges in using them, for example, being unaware of certain pythonic idioms
or do not know how to use them properly. Based on an analysis of 7,638 Python
repositories on GitHub, we find that non-idiomatic Python code that can be
implemented with pythonic idioms occurs frequently and widely. Unfortunately,
there is no tool for automatically refactoring such non-idiomatic code into
idiomatic code. In this paper, we design and implement an automatic refactoring
tool to make Python code idiomatic. We identify nine pythonic idioms by
systematically contrasting the abstract syntax grammar of Python and Java. Then
we define the syntactic patterns for detecting non-idiomatic code for each
pythonic idiom. Finally, we devise atomic AST-rewriting operations and
refactoring steps to refactor non-idiomatic code into idiomatic code. We test
and review over 4,115 refactorings applied to 1,065 Python projects from
GitHub, and submit 90 pull requests for the 90 randomly sampled refactorings to
84 projects. These evaluations confirm the high-accuracy, practicality and
usefulness of our refactoring tool on real-world Python code. Our refactoring
tool can be accessed at 47.242.131.128:5000.Comment: 12 pages, accepted to ESEC/FSE'202
Towards Responsible AI in the Era of ChatGPT: A Reference Architecture for Designing Foundation Model-based AI Systems
The release of ChatGPT, Bard, and other large language model (LLM)-based
chatbots has drawn huge attention on foundations models worldwide. There is a
growing trend that foundation models will serve as the fundamental building
blocks for most of the future AI systems. However, incorporating foundation
models in AI systems raises significant concerns about responsible AI due to
their black box nature and rapidly advancing super-intelligence. Additionally,
the foundation model's growing capabilities can eventually absorb the other
components of AI systems, introducing the moving boundary and interface
evolution challenges in architecture design. To address these challenges, this
paper proposes a pattern-oriented responsible-AI-by-design reference
architecture for designing foundation model-based AI systems. Specially, the
paper first presents an architecture evolution of AI systems in the era of
foundation models, from "foundation-model-as-a-connector" to
"foundation-model-as-a-monolithic architecture". The paper then identifies the
key design decision points and proposes a pattern-oriented reference
architecture to provide reusable responsible-AI-by-design architectural
solutions to address the new architecture evolution and responsible AI
challenges. The patterns can be embedded as product features of foundation
model-based AI systems and can enable organisations to capitalise on the
potential of foundation models while minimising associated risks
Prompt Sapper: LLM-Empowered Software Engineering Infrastructure for AI-Native Services
Foundation models, such as GPT-4, DALL-E have brought unprecedented AI
"operating system" effect and new forms of human-AI interaction, sparking a
wave of innovation in AI-native services, where natural language prompts serve
as executable "code" directly (prompt as executable code), eliminating the need
for programming language as an intermediary and opening up the door to personal
AI. Prompt Sapper has emerged in response, committed to support the development
of AI-native services by AI chain engineering. It creates a large language
model (LLM) empowered software engineering infrastructure for authoring AI
chains through human-AI collaborative intelligence, unleashing the AI
innovation potential of every individual, and forging a future where everyone
can be a master of AI innovation. This article will introduce the R\&D
motivation behind Prompt Sapper, along with its corresponding AI chain
engineering methodology and technical practices
Trust in Software Supply Chains: Blockchain-Enabled SBOM and the AIBOM Future
Software Bill of Materials (SBOM) serves as a critical pillar in ensuring
software supply chain security by providing a detailed inventory of the
components and dependencies integral to software development. However,
challenges abound in the sharing of SBOMs, including potential data tampering,
hesitation among software vendors to disclose comprehensive information, and
bespoke requirements from software procurers or users. These obstacles have
stifled widespread adoption and utilization of SBOMs, underscoring the need for
a more secure and flexible mechanism for SBOM sharing. This study proposes a
novel solution to these challenges by introducing a blockchain-empowered
approach for SBOM sharing, leveraging verifiable credentials to allow for
selective disclosure. This strategy not only heightens security but also offers
flexibility. Furthermore, this paper broadens the remit of SBOM to encompass AI
systems, thereby coining the term AI Bill of Materials (AIBOM). This extension
is motivated by the rapid progression in AI technology and the escalating
necessity to track the lineage and composition of AI software and systems.
Particularly in the era of foundational models like large language models
(LLMs), understanding their composition and dependencies becomes crucial. These
models often serve as a base for further development, creating complex
dependencies and paving the way for innovative AI applications. The evaluation
of our solution indicates the feasibility and flexibility of the proposed SBOM
sharing mechanism, positing a new solution for securing (AI) software supply
chains
- …